Some Proofs on Statistical Magnitudes for Continuous Phenomena 
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Abstract: In this work, the proofs concerning the continuity of the disequilibrium. Shannon information and statistical 
complexity in the space of distributions are presented. Also, some results on the existence of Shannon information for 
continuous systems are given. 
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1 Introduction 

The concepts of Shannon information (or entropy), S, 
and disequilibrium, D, for a continuous system are 
given by 



S 
D 



-k 



p{x) log p{x) dx . 



p^{x) dx , 



(1) 



(2) 



where A; is a positive constant (that is chosen to be 
equal to I), x represents the continuum of the sys- 
tem states, and p{x) stands for the normahzed density 
function of all these states. 

The statistical complexity C, the so-called LMC 
complexity Ql, is defined as 



C = H-D 



(3) 



where H gives account of the broadness of the distri- 
bution and and D gives an idea of how much spiky 
is the distribution respect to equilibrium distribution, 
that in this case is the equiprobability. For our pur- 
pose, we take a version used in Ref. [2 1 as quantifier 
of H. This is the simple exponential Shannon entropy 
||3]| . that takes the form. 



H 



(4) 



that implies the positivity of H and C. These informa- 
tion theoretic indicators has been successfully applied 
in different contexts in order to unveil a complex be- 
havior or structure, for instance, in gases out of equi- 
librium [4], in coupled map lattices [5J or in quantum 
systems IS, to cite some of them. 

In this communication, it is our objective to prove 
rigorously that all these functionals, H, D and C, are 



near-continuous, i.e. similar distributions take a simi- 
lar value of information, disequilibrium and statistical 
complexity. Also, we present some exact results on 
the values that Shannon information can take for con- 
tinuous systems. 

2 Continuity of D and C 

First, we remind some definitions that were used in 
Ref. la. 

Definition 1 . Let I be an interval in IR and 5 > 
We will say that two density functions, namely f and g, 
are 6-neighboring functions on I if both are supported 
on I and the Lebesgue measure of the set of points of I 
such that \f(x) — g{x)\ > 6 is zero, i.e. if the essential 
supremun (ess sup) of\f — g\ verifies 

ess sup \f{x) — g{x)\ < 5. 

Note that this definition can be applied to non bounded 
density functions. 

Definition 2 . We will say that a real map T defined 
on density functions is near-continuous on I if for any 
£ > there exists 6{£,I) > such that if f and g 
are two d-neighboring functions on I then \T{f) — 
T{g)\<e. 

Tlieorem 3 . Disequilibrium D is near-continuous 
on ]R. 

Proof. Take e > 0, 5 < f . Let /, 5 be two 
(^-neighboring functions on IR. Then, 



D{f)-D{g) 



{f\x)-g\x))dx (5) 



ifix)+g{x))* 
(fix) - g{x))dx. 



As f, g are (5-neighboring functions on ]R we have 
that |/(a:) — g{x)\ < 6 a.e. and so, 



\D{f)-Dig)\ < 



< 



ifix)+gix))* 
\f{x) - g{x)\dx 
f{x) dx+ 



— oo 

CXI 



25 <e. 



g{x) dx 
4 



Note that if we consider only bounded den- 
sity functions, then D is a continuous map between 
normed spaces in the topological sense. 

In order to prove the near-continuity of the Shan- 
non information, S, and for extension of its exponen- 
tial, in this case, H, the following technical Lemma is 
needed. 

Lemma 4 . For all a > 1 there exists (5q, < 1 such 
that for any pair of points x\, x^ with < xi, X2 < a 
and \x\ — X2\ < b < ba_ we have that 

l^ilogxi — rE2logX2| < |51og5| . 

Proof. Given b € (0, 1) consider the function 

si,{x) = (x + (5)log(x + 5) — xlogx, 

which can be extended to x = with continuity and 



is derivable in 1R+ with 



SAX] 



log^. As (5 > 0, 



X + 5 > X and so > 0. This means that is 
rising on 1R+. Observe that as 5 < 1, log(5 < and 
so 5^(0) < 0. On the other hand, as is rising and 
a > 1, we have that < (l + (^)log(l + (5) = s^iX) < 
ss{a). This means that there is only one unique xq so 
that ss{x) < for any < x < xq and ss{x) > for 
the values of xq < x < a. 

The result of the Lemma, expressed in terms of 
the function sg, is equivalent to finding 6a such that 
for any < 6 < 6a and < x < a, 

\ss{x)\ < \ssiO)\. 

Clearly, this inequality is verified for any x < xq. 
So, we can suppose x > xq. In this case the inequality 
can be written as 

ssix) < -ss{0). 



On the other hand, as ss is rising, it can be seen 
that the verification of the former inequality is equiv- 
alent to the verification of the next one: 

ss{a) < -5^(0). 

Let us consider the function Fa{6) = ss{a) + 
ss{0)- The problem has been reduced to finding a 6a 
such that for all < 6 < 6a, Fa{6) < 0. Since 
F^{6) = 2 + log{6{a + 6)), we will take 6a as the 
only positive number such that F^{6) = 0, then 



1 

2^ 



ae + J{aeY + 4 



As Fa is decreasing on [0, 5a], rising on [6a, oo) and 
Fa < 0, we can conclude that there exists an unique 
5a bigger than 6a such that for all < 6 < 6a, 

Fa{6) < 0. « 

Theorem 5 . H is near-continuous on every compact 
K (IM. 

Proof. First, we will prove that S is near- 
continuous. Then, by continuity of the exponential 
operation, H will also be near-continuous. 

Let K he. n compact set in IR. Take e < 1, 5 > 
with 5 < min{(52, |}, where 62 is the 5a given in 
Lemma|4]for a = 2, and |(51og(5| < where \K\ 
stands for the Lebesgue measure of K. 

Let / and g be two (^-neighboring functions on K 
and let us consider the sets 

A = {x G K; /(x) > 5(x) > 1}, 

B = {x G K; g{x) > /(x) > 1}, 

C = {x^K- f{x)=g{x)], 

D = {xeK; f{x)<2,g{x)<2}. 

As6 < 1, AU BUCU D = M. Defining G = 
/ log / - 5 log g, we have 



\S{f)-S{g)\ < 



< 



\G{x)\dx 
|G(x)|dx + 



B 



(6) 



\G{x)\dx + 



\G{x)\dx+ / \G{x)\dx. 

C JD 

We will bound separately each one of the above 
integrals. Note that the third one is null and the first 
two ones behave in a similar manner. 

Applying the Mean Value Theorem to the func- 
tion s{t) = t log(t) we have that for any x G A there 
exists X such that g{x) < x < /(x) and 

G(x) = [fix)-gix)]s\S:) 

= [f {x) - g{x)][l + logx] . 



As X G ^, 1 < g{x). Then, 1 < x < f{x) and 
so logx < X < f{x). From here we conclude that 
1 + log x < f{x) +g{x) and then the first integral can 
be bounded as 

/ \G{x)\dx < [ \f{x) - g {x)\[l + log S:]dx 
J A J A 

< S f {f{x)+g{x))dx = 26 < ^. 

Analogously, 

JjG{x)\dx <|. 

For the last integral we make use of the Lemma 
m As (5 < (^2, we have that 

\G{x)\ = \flogf - glogg\ < \6log6\. 

And as 6 has been chosen so that \6 log 6\ < 

[ \G{x)\dx < - . 
Jd 2 

By putting together in Q the inequalities ob- 
tained for each integral, then finally it is proved that 
S is near-continuous. Hence, by extension, H is also 
near-continuous and the Theorem [5] is proved. Jl» 

Corollary 6 . The complexity C is near-continuous 
on compacts. 

Proof. It is immediate from the definition of C 
and Theorems [3] and m 4> 

As it was observed in Ref. 12], let us remark the 
necessity to have compactness in the sets K in order 
to prove the near-continuity of H. The same can be 
said for C. 

3 On the Existence of Entropy 

Here, we proceed to show some results on the exis- 
tence of the Shannon entropy S for continuous sys- 
tems. 

Proposition 7 . Given a density function p{x) 
bounded and compactly supported, its entropy S{p) 
is finite. 

Proof. Let G be the upper bound of p{x). As 
< p < C on the support K of p, and the function 
X log X has range in [— , oo) for x > 0, then 

-- KK) < -S{p) < max{0, G log G} /i(A') . 
e 

So the integral of the entropy is convergent. X 

Let us observe that if p{x) is not bounded or has 
no bounded support, this proposition may not be truth. 
In this situation, it will be possible to have one of the 
following cases: 



• (I) S{p) does exist and is finite, 

• (2) S{p) does exist and is +oo, 

• (3) S{p) does exist and is — oo, 

• (4) S{p) does not exist. 

All these cases are possible, as it can be seen with 
the following example. Take p{x) = - supported on 
different sets in 1R+. Without loss of generality it will 
be assumed that the support of the density function 
p{x) is a set B consisting on a countable union of in- 
tervals in 1R+. 

The following notation: B = B^ U B^ = 
(UneiN In) U (UneiN ^n) IS uscd, whcrc 

In = {ln,l'n) Verifies 

1 < 70 < To < • • • < 7n < 7; < • • • 

and Jn = {Vn,Vn) verifies 

1 >r/[, >r?o > ••• >??n >r/n > ••• >0. 

Now we consider the projection of such intervals 
onto IR through the logarithmic function. Let us call 

Tn = {Tn,Tn) = log /„ and Sn = {cTn^CTn) = log Jn 

and so . . . < (T„ < (7^ < . . . < (To < (Tq < < 
To < To < . . . < T„ < r; < . . .. We will call A = 
(UneiN Tn) U (U„6]N Sn) = A+ U A' , and introduce 
the notation 5n = t'^ — Tn and pn = a'^ — an- 

It is easy to see that p{x) = -Xb{x) is of density 
1 if and only if E^=o i^n + Pn) = 1- 

Let p be the Lebesgue measure. Let us define 

ff^\ ^ f p{y > x;y e A} X > 0, 
{-fJ-iy < x;y e A} x < 0, 

and observe that 

lim /(x) - lim /(x) = p{A+) + p{A-) = 1 . 

a:^0+ x~)-0- 

Theorem 8 . Let the former p{x) = ^Xb{x) be a 
density function. Then, 

1. If f is integrable Riemann on JR (f S TZ^JR)) 
then S{p) € IR. Besides, the finite value of S{p) 
is given by 

f{x)dx= / f{x)dx+ f{x)dx. 
-00 Jtr- Ju+ 

2. If the value of the integral of f in IR^ is finite 
and in IR^ is infinite, i.e. f G TZ(iR~) \7^(IR~''), 
then S{p) = +00 . 

3. If the value of the integral of f in IR is infinite 
and in 1R+ is finite, i.e. f G 7^(lR+) \ 7^(1R~), 
then S{p) = —00 . 



4. If f ^ n{M-) U 7e(lR+) then S{p) does not 
exist . 

Proof. First, we see that 



oo roo 

p{x)logp{x)dx = / f{x)dx. (7) 
1 Jo 



In fact, 



p{x) logp{x)dx 



log(x) 



dx 



B+ X 

oo 



E / ^^dx 



n=0 
1 oo 



^log^(x)| 



n=0 



-/2 _2n 



■EC 

n=0 
oo 

n=0 



By other side, analyzing the function / on IR+, 
we have that Vx G [0, tq], /(x) = and that 

\/x G [r|5,ri], /(x) = - (5o. Also, Vx G 

['n3,r^], /(x) = n,{A+) - (x - To), and so on. It 
is also known that X^nGiN = /^(^^)- Hence, 

/ /(x)dx = Y.{5nrn + -5l) 

oo -j^ 



n=0 

oo 



n=0 



^7 E ^"("^n 



n=0 



This proves (|7]). 

Similarly, it can be proved that 



1 

p{x) log p{x)dx = / f{x)dx. (8) 

^-oo 



The result of the theorem is obtained immediately 
from © and dH). X 



are continuous magnitudes when applied to continu- 
ous phenomena. In this case, it has been put in evi- 
dence that the property of compactness for the support 
of the density distribution is crucial. Also, some ex- 
act results on the possible values that entropy can take 
have been presented. 
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4 Conclusion 

Continuity is a desirable property for any good be- 
haved magnitude, although it does not imply the trivi- 
ality of its mathematical proof. Here, we have shown 
that entropy, disequilibrium and statistical complexity 



